Non-Redundant Spectral Dimensionality Reduction

摘要

Spectral dimensionality reduction algorithms are widely used in numerousdomains, including for recognition, segmentation, tracking and visualization.However, despite their popularity, these algorithms suffer from a majorlimitation known as the "repeated Eigen-directions" phenomenon. That is, manyof the embedding coordinates they produce typically capture the same directionalong the data manifold. This leads to redundant and inefficientrepresentations that do not reveal the true intrinsic dimensionality of thedata. In this paper, we propose a general method for avoiding redundancy inspectral algorithms. Our approach relies on replacing the orthogonalityconstraints underlying those methods by unpredictability constraints.Specifically, we require that each embedding coordinate be unpredictable (inthe statistical sense) from all previous ones. We prove that these constraintsnecessarily prevent redundancy, and provide a simple technique to incorporatethem into existing methods. As we illustrate on challenging high-dimensionalscenarios, our approach produces significantly more informative and compactrepresentations, which improve visualization and classification tasks.